2022-04-18 12:53:53

Acknowledgements

  • Support from NICHD, NIH OD, NIMH, NIDA, OBSSR, NSF
  • Karen Adolph, Cathie Tamis-LeMonda, Orit Hertzberg, Tiger Teng

Overview

  • What is PLAY
  • Why PLAY
  • Challenges..met and meeting
  • A preview
  • Let play be our work

What is PLAY

  • ~1,000 mother-infant dyads (12-, 18-, 24-mos)
  • 1 hr natural behavior (video)
  • 5 min structured play (video)
  • House tour (video)
  • Parent-report questionnaires
  • Ambient sound levels

  • Foundational video coding passes
    • Speech & language
    • Emotional expression
    • Object interaction
    • Locomotion & physical activity

  • Common, openly shared dataset
  • Dataset as deliverable
  • Rigorous QA control
  • Investigator-specific questions…
  • Catalyze/expand capacity to exploit video
  • Pioneer & polish tools for big data open developmental science

Why PLAY

Natural behavior

Big data developmental science

Assuming a realistic range of prior probabilities for null hypotheses, false report probability is likely to exceed 50% for the whole literature.

Open developmental science

The advancement of detailed and diverse knowledge about the development of the world’s children is essential for improving the health and well-being of humanity. The Society for Research in Child Development (SRCD) regards scientific integrity, transparency, and openness as essential for the conduct of research and its application to practice and policy. These values apply to research conduct, to the teaching of scientific methods, and to the translation of science into practice and policy.

Open developmental science

PLAY: Science in the open

Growing Databrary

Video, audio, + other types

Challenges…met and meeting

Sampling

  • Who to include
  • Age range(s)
  • What language backgrounds
  • Where to sample
  • What to vary, what to keep constant…

  • Is ‘an hour in the life’ representative?
  • What parent-report survey questions & how to collect
  • What behaviors to code from video

  • 12-mo-olds, 18-mo-olds, & 24-mo-olds
  • Only English and Spanish-speaking households
  • Mom and child
  • 31 sites

Races reported in PLAY counties

Races reported in PLAY counties

Ethnicity reported in PLAY counties

Ethnicity reported in PLAY counties

Languages spoken in PLAY counties

Languages spoken in PLAY counties

Educational attainment in PLAY counties

Educational attainment in PLAY counties

Household income in PLAY counties

Household income in PLAY counties

Survey questions

  • Health
  • Patient Health Questionnaire (PHQ-4)
  • Locomotor milestones
  • MacArthur-Bates Communicative Development Inventory (MB-CDI)

  • Early Childhood Behavior Questionnaire (Rothbart)
  • Media use
  • Pets
  • Household structure
  • Typical day

Behaviors to code

Other challenges

  • Findable, usable beyond launch group
  • Limitations of Databrary 1.0
    • Data in-process vs. data shared
    • Virtual volumes
  • Versioning data, protocol, coding schemes
  • IRB

  • Add-on, follow-up studies
    • New data
    • Augmented, new video annotations
  • COVID-19

A preview

Release levels

Video data

Survey data

  • Make use of databraryapi R package that interacts with Databrary API
  • Reproducible data cleaning, visualization, and analysis scripts from the get-go

databraryapi::login_db("myemail@university.edu")

play_data <- databraryapi::read_csv_data_as_df(session_id = 51539, asset_id = 366382)

Demographics

xtabs(formula = ~ child_sex + age_group, data = play_data) %>%
  knitr::kable(., format = 'html') %>%
  kableExtra::kable_classic(.)
12mo 18mo 24mo
female 12 13 6
male 8 11 18

xtabs(formula = ~ child_race + child_ethnicity, data = play_data) %>%
  knitr::kable(., format = 'html') %>%
  kableExtra::kable_classic(.)
Hispanic or Latino Not Hispanic or Latino
Asian 0 1
Black or African American 1 0
More than one 9 6
Other 0 2
White 4 45

Virtues of standardization within a repository

  • Databrary has demographic data from \(n=9,342\) participant sessions.
  • Race (NIH categories), Ethnicity (NIH categories), Gender
  • Building foundation for future searching and filtering by participant characteristics

Locomotor milestones

Feeding

Typical behavior?

xtabs(formula = ~ typical_behavior + age_group, data = typical_data) %>%
  knitr::kable(., format = 'html') %>%
  kableExtra::kable_classic(.)
12mo 18mo 24mo
no 1 5 0
yes 18 18 24

Typical night/morning?

xtabs(formula = ~ typical_nightmorning + age_group, data = typical_data) %>%
  knitr::kable(., format = 'html') %>%
  kableExtra::kable_classic(.)
12mo 18mo 24mo
no 6 3 3
yes 13 21 21
## [1] TRUE

Making play our work

  • Big data developmental science of natural behavior is possible & necessary
  • Cognition and emotion in context
  • Ambition and vision drive innovation
  • Future-orientation (what will future researchers want to know) challenging, but invigorating
  • What do we want our science to be about?

Come PLAY with us!

  • Collaborate with a launch group member
  • Write grants to use the data
  • Help test, polish the databraryapi R package and complete the Python package
  • Help shape, test Databrary 2.0
  • Make developmental science a leader in big data reseearch on behavior

Resources

This talk was produced on 2022-04-18 in RStudio using R Markdown and the ioslides framework. The code and materials used to generate the slides may be found at https://github.com/PLAY-behaviorome/2022-04-21-team-sci-cds/. Information about the R Session that produced the code is as follows:

## R version 4.1.2 (2021-11-01)
## Platform: x86_64-apple-darwin17.0 (64-bit)
## Running under: macOS Monterey 12.3
## 
## Matrix products: default
## LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
## 
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
## [1] forcats_0.5.1   stringr_1.4.0   dplyr_1.0.8     purrr_0.3.4    
## [5] readr_2.1.1     tidyr_1.1.4     tibble_3.1.6    ggplot2_3.3.5  
## [9] tidyverse_1.3.1
## 
## loaded via a namespace (and not attached):
##  [1] httr_1.4.2         sass_0.4.1         jsonlite_1.8.0    
##  [4] viridisLite_0.4.0  splines_4.1.2      modelr_0.1.8      
##  [7] bslib_0.3.1        shiny_1.7.1        assertthat_0.2.1  
## [10] highr_0.9          cellranger_1.1.0   yaml_2.3.5        
## [13] lattice_0.20-45    pillar_1.7.0       backports_1.4.1   
## [16] glue_1.6.2         digest_0.6.29      promises_1.2.0.1  
## [19] rvest_1.0.2        colorspace_2.0-3   Matrix_1.3-4      
## [22] htmltools_0.5.2    httpuv_1.6.5       pkgconfig_2.0.3   
## [25] broom_0.7.11       haven_2.4.3        xtable_1.8-4      
## [28] scales_1.2.0       webshot_0.5.2      svglite_2.0.0     
## [31] later_1.3.0        tzdb_0.2.0         mgcv_1.8-38       
## [34] farver_2.1.0       generics_0.1.2     ellipsis_0.3.2    
## [37] withr_2.5.0        cli_3.2.0          magrittr_2.0.3    
## [40] crayon_1.5.1       readxl_1.3.1       mime_0.12         
## [43] evaluate_0.15      fs_1.5.2           fansi_1.0.3       
## [46] nlme_3.1-153       xml2_1.3.3         tools_4.1.2       
## [49] hms_1.1.1          lifecycle_1.0.1    munsell_0.5.0     
## [52] reprex_2.0.1       kableExtra_1.3.4   compiler_4.1.2    
## [55] jquerylib_0.1.4    systemfonts_1.0.3  rlang_1.0.2       
## [58] grid_4.1.2         rstudioapi_0.13    miniUI_0.1.1.1    
## [61] labeling_0.4.2     rmarkdown_2.13     gtable_0.3.0      
## [64] DBI_1.1.2          curl_4.3.2         R6_2.5.1          
## [67] lubridate_1.8.0    knitr_1.38         keyring_1.3.0     
## [70] fastmap_1.1.0      databraryapi_0.2.8 utf8_1.2.2        
## [73] ggExtra_0.9        stringi_1.7.6      Rcpp_1.0.8.3      
## [76] vctrs_0.4.1        dbplyr_2.1.1       tidyselect_1.1.2  
## [79] xfun_0.30

References

Adolph, K. E. (2020). Oh, behave!: PRESIDENTIAL ADDRESS, XXth international conference on infant studies new orleans, LA, US may 2016. Infancy: The Official Journal of the International Society on Infant Studies, 25(4), 374–392. https://doi.org/10.1111/infa.12336

Adolph, K.E., Gilmore, R.O., & Kennedy, J.L. (2017). Video as data and documentation will improve psychological science. https://www.apa.org/science/about/psa/2017/10/video-data. Retrieved from https://www.apa.org/science/about/psa/2017/10/video-data

Gennetian, L. A., Tamis‐LeMonda, C. S., & Frank, M. C. (2020). Advancing transparency and openness in child development research: opportunities. Child Development Perspectives, 14(1), 3–8. https://doi.org/10.1111/cdep.12356

Gilmore, R. O. (n.d.). Databraryapi. Github. Retrieved from https://github.com/PLAY-behaviorome/databraryapi

Gilmore, R. O. (2016). From big data to deep insight in developmental science. Wiley Interdisciplinary Reviews: Cognitive Science, 7(2), 112–126. https://doi.org/10.1002/wcs.1379

Gilmore, R., & Adolph, K. E. (2017). Video can make behavioural research more reproducible. Nature Human Behavior, 1. https://doi.org/10.1038/s41562-017-0128

Gilmore, R., Cole, P., Verma S, Aken, Marcel A, & Worthman, C. (2020). Advancing scientific integrity, transparency, and openness in child development research: Challenges and possible solutions. Child Development Perspectives, 14(1), 9–14. https://doi.org/10.1111/cdep.12360

Gilmore, R., & Qian, Y. (2021). An open developmental science will be more rigorous, robust, and impactful. Infant and Child Development. https://doi.org/10.1002/icd.2254

Marek, S., Tervo-Clemmens, B., Calabro, F. J., Montez, D. F., Kay, B. P., Hatoum, A. S., … Dosenbach, N. U. F. (2022). Reproducible brain-wide association studies require thousands of individuals. Nature, 603(7902), 654–660. https://doi.org/10.1038/s41586-022-04492-9

Soska, K. C., Xu, M., Gonzalez, S. L., Herzberg, O., Tamis-LeMonda, C. S., Gilmore, R. O., & Adolph, K. E. (2021). (Hyper)active data curation: A video case study from behavioral science. Journal of Escience Librarianship, 10(3). https://doi.org/10.7191/jeslib.2021.1208

Szucs, D., & Ioannidis, J. P. A. (2017). Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature. PLoS Biology, 15(3), e2000797. https://doi.org/10.1371/journal.pbio.2000797

Yarkoni, T. (2020). The generalizability crisis. The Behavioral and Brain Sciences, 1–37. https://doi.org/10.1017/S0140525X20001685